Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
Haptic rendering is the translation of forces in a virtual environment to a physical “haptic” device that can provide touch-based, a.k.a. haptic, feedback to a user of the haptic device. Both impedance type and admittance type haptic devices are available.
To provide haptic feedback about the virtual environment, objects in the virtual environment are often represented as a collection of polygons, such as triangles, that can be operated upon using a haptic device. The haptic device can be controlled using a “Haptic Interaction Point” (HIP) in the virtual environment, which performs a similar function for the haptic device as a mouse pointer does for a computer mouse. Ideally, the HIP should not be able to penetrate virtual environment objects.
FIGS. 1A-1C depict a scenario 100 that illustrates a prior art technique of utilizing proxy 130 to control interactions between HIP 110 and polygon 120 in a virtual environment. In scenario 100, HIP 110 starts as the position shown as HIP 110a in FIG. 1A, moves through position HIP 110b shown in FIG. 1B, and ends at position HIP 110c shown in FIG. 1C. In this technique, HIP 110 and proxy 130 are connected by a simulated spring not shown in the Figures.
In FIG. 1A, proxy 130, shown at position 130a, is in “free motion”; e.g., proxy 130a is not touching polygon 120. In FIG. 1B, proxy 130, shown at position 130b, is “in contact” with polygon 120. In scenario 100, while HIP 110 continues to move down from position 110b into polygon 120, proxy 130 is not permitted to enter into polygon 120. In FIG. 1C, proxy 130, shown at position 130c, is still in contact with a surface of polygon 120 after HIP 110 has moved to position 110c inside of polygon 120. As the distance increases between HIP 110 and proxy 130, the simulated spring exerts a proportionally larger force to draw HIP 110 closer to proxy 130. FIG. 1C shows the simulated spring force as force 140 exerted on HIP 110. As shown in FIG. 1C, force 140 is exerted in the direction of a normal of the surface in contact with the proxy 130; e.g., a hypotenuse 122 of polygon 120.
Recently, relatively inexpensive, low-latency cameras have been made commercially available that provide both image and depth information about a real environment. One example is the camera used by KINECT® for XBOX 360® by the Microsoft Corporation of Redmond, Wash. The KINECT® camera includes an RGB camera and a depth sensor. The KINECT® camera can capture and provide RGB+D data, or video (RGB) and depth (D) information, at a 30 Hertz frame rate. The RGB+D data provided by the KINECT® uses 8-bit VGA resolution (640×480 pixels) for an RGB video stream and one 11-bit depth value per pixel as a monochrome depth sensor data stream.
In the realm of robotic surgery, the introduction of the da Vinci surgical robot (Intuitive Surgical, Inc., Sunnyvale, Calif.) in 2001 provided an additional laparoscopic technological adjunct which has rapidly penetrated the surgical arena. Adapted from research exploring remote surgical capabilities for the military, the da Vinci robot is the only commercially available remote surgical robot. The fields of general surgery, urology, gynecology, cardiothoracic surgery, pediatric surgery and otolaryngology have to varying degrees embraced this technology and its adoption is evidenced particularly by the rapid increase in the number of prostatectomies for prostate cancer.
Even though the da Vinci has dramatically enhanced the dexterity of laparoscopic procedures, the integration between computation and surgery is essentially unchanged from prior practice. In current robotic surgery, the robot is merely an extension of existing tools. In particular, the current state-of-the-art for visualization during robotic surgery involves a rigid single view 3D endoscope.
Other environments can be explored by robots, such as undersea, outer space, and hazardous environments. In some of these environments, robots can be controlled by human operators receiving video and/or audio information from the robot.